1 Stroke vs. Bleeding Shiny App Design Proposal

Stroke vs. Bleeding Shiny App Design Proposal # To Do 1. Work with a stats student (Yayie Duan) to write a Shiny app.

  1. After meeting with Javier (01/28/2021):
  1. Instead of linear regression, use Deming regression
  2. Probability of being in the green region based on the confidence intervals (taken care of Deming regression?)
  3. Think about: ratio of variance (related to Deming regression)
  1. After meeting with Cande (01/28/2021):
  1. Add more variables to the model. I’m thinking of adding CHADSVASC variables and displaying the score additional to the recommendation. We can also extend it to a more comprehensive model and use the app to also demonstrate how the MIDAS-based risk calculator compares to CHADSVASC risk estimation.
    b. IF we are going to assign probabilities based on confidence intervals (see ‘b’ above), we can have an internal rule, e.g. if p > 0.5 then medicate, if < then don’t). This does not need to be displayed in the app to make it less confusing for the patients.

2 Setup

require(data.table)
Loading required package: data.table
Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     
data.table 1.13.6 using 2 threads (see ?getDTthreads).  Latest news: r-datatable.com
require(ggplot2)
Loading required package: ggplot2
require(plotly)
Loading required package: plotly
Registered S3 methods overwritten by 'htmltools':
  method               from         
  print.html           tools:rstudio
  print.shiny.tag      tools:rstudio
  print.shiny.tag.list tools:rstudio
Registered S3 method overwritten by 'htmlwidgets':
  method           from         
  print.htmlwidget tools:rstudio

Attaching package: 㤼㸱plotly㤼㸲

The following object is masked from 㤼㸱package:ggplot2㤼㸲:

    last_plot

The following object is masked from 㤼㸱package:stats㤼㸲:

    filter

The following object is masked from 㤼㸱package:graphics㤼㸲:

    layout
require(deming)
Loading required package: deming
require(MethComp)
Loading required package: MethComp

ATTENTION! In ggplot, must explicitly set ‘inherit.aes = FALSE’, otherwise ‘geom_ribbon’ expects values for aesthetics from previous geoms (i.e. tt variables ps, pb, etc.). See solution here:
(geom_ribbon expects an unused y-axis variable to be present)[https://github.com/tidyverse/ggplot2/issues/3364]

3 Load data

load("data/tt.RData")

4 Metod 1: Linear regression

4.1 Run ‘lm’

# Log2-transformed variables
tt[, c("l2pb",
       "l2ps") := list(log2(pb), #xx
                        log2(ps))] #yy
# Regression on log2-transformed variables
m2 <- lm(l2ps ~ l2pb,
         data = tt)
summary(m2)

Call:
lm(formula = l2ps ~ l2pb, data = tt)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.67656 -0.50577  0.02698  0.40965  0.72161 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -4.5353     1.4238  -3.185  0.00973 ** 
l2pb          1.4596     0.1978   7.379 2.37e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.553 on 10 degrees of freedom
Multiple R-squared:  0.8449,    Adjusted R-squared:  0.8293 
F-statistic: 54.45 on 1 and 10 DF,  p-value: 2.371e-05
new_l2pb <- seq(from = 5.5,
                to = 8.75,
                length.out = nrow(tt))
# new_l2pb <- seq(from = 5, 
#                 to = 9,
#                 length.out = nrow(tt))
pi_20_80 <- data.table(predict(m2, 
                               newdata = list(l2pb = new_l2pb),
                               interval = "prediction",
                               level = 0.6))
pi_20_80$l2pb <- new_l2pb
head(pi_20_80)

4.2 Plot Fear Bleeding 20 on 0-100 Scale (Log2)

p1 <- ggplot(tt,
             aes(x = l2pb,
                 y = l2ps,
                 fill = lab)) +
  geom_errorbar(aes(ymin = log2(10^4*(prob.stroke - 1.96*se.stroke)),
                    ymax = log2(10^4*(prob.stroke + 1.96*se.stroke))),
                size = 0.5) +
  geom_errorbar(aes(xmin = log2(10^4*(prob.bleed - 1.96*se.bleed)),
                    xmax = log2(10^4*(prob.bleed + 1.96*se.bleed))),
                size = 0.5) +
  geom_point(shape = 21,
             aes(size = sqrt(N))) +
  geom_abline(slope = m2$coefficients[2],
              intercept = m2$coefficients[1],
              linetype = "dashed") +
  geom_ribbon(data = pi_20_80,
              aes(x = l2pb,
                  ymin = lwr,
                  ymax = 9.25),
              fill = "green",
              alpha = 0.2,
              inherit.aes = FALSE) + 
  scale_x_continuous("Log2 of Bleeding per 10^4 patients",
                      # limits = c(5.5, 8.5),
                     expand = c(0, 0)) +
  scale_y_continuous("Log2 of Stroke per 10^4 patients",
                     # limits = c(3, 9),
                     expand = c(0, 0)) +
  ggtitle("Fear of Bleeding: 20/100\nGREEN = MEDICATE") +
  theme_bw() +
  theme(plot.title = element_text(hjust = 0.5))
ggplotly(p1)

4.3 Plot Fear Bleeding 80 on 0-100 Scale (Log2)

p2 <- ggplot(tt,
             aes(x = l2pb,
                 y = l2ps,
                 fill = lab)) +
  geom_errorbar(aes(ymin = log2(10^4*(prob.stroke - 1.96*se.stroke)),
                    ymax = log2(10^4*(prob.stroke + 1.96*se.stroke))),
                size = 0.5) +
  geom_errorbar(aes(xmin = log2(10^4*(prob.bleed - 1.96*se.bleed)),
                    xmax = log2(10^4*(prob.bleed + 1.96*se.bleed))),
                size = 0.5) +
  geom_point(shape = 21,
             aes(size = sqrt(N))) +
  geom_abline(slope = m2$coefficients[2],
              intercept = m2$coefficients[1],
              linetype = "dashed") +
  geom_ribbon(data = pi_20_80,
              aes(x = l2pb,
                  ymin = upr,
                  ymax = 9.25),
              fill = "green",
              alpha = 0.2,
              inherit.aes = FALSE) + 
  scale_x_continuous("Log2 of Bleeding per 10^4 patients",
                     expand = c(0, 0)) +
  scale_y_continuous("Log2 of Stroke per 10^4 patients",
                     expand = c(0, 0)) +
  ggtitle("Fear of Bleeding: 80/100\nGREEN = MEDICATE") +
  theme_bw() +
  theme(plot.title = element_text(hjust = 0.5))
ggplotly(p2)

4.4 Readjust predictions

new_l2pb <- seq(from = 5,
                to = 9,
                length.out = 10*nrow(tt))
pi_20_80 <- data.table(predict(m2, 
                               newdata = list(l2pb = new_l2pb),
                               interval = "prediction",
                               level = 0.6))
pi_20_80$l2pb <- new_l2pb
head(pi_20_80)

4.5 Plot Fear Bleeding 20 on 0-100 Original Scale

p1 <- ggplot(tt,
             aes(x = pb,
                 y = ps,
                 fill = lab)) +
  geom_errorbar(aes(ymin = 10^4*(prob.stroke - 1.96*se.stroke),
                    ymax = 10^4*(prob.stroke + 1.96*se.stroke)),
                size = 0.5) +
  geom_errorbar(aes(xmin = 10^4*(prob.bleed - 1.96*se.bleed),
                    xmax = 10^4*(prob.bleed + 1.96*se.bleed)),
                size = 0.5) +
  geom_point(shape = 21,
             aes(size = sqrt(N))) +
  geom_ribbon(data = pi_20_80,
              aes(x = 2^l2pb,
                  ymin = 2^lwr,
                  ymax = 400),
              fill = "green",
              alpha = 0.2,
              inherit.aes = FALSE) + 
  geom_line(data = pi_20_80,
            aes(x = 2^l2pb,
                y = 2^fit),
            linetype = "dashed",
            inherit.aes = FALSE) +
  scale_x_continuous("Bleeding per 10^4 patients",
                     limits = c(50, 400),
                     expand = c(0, 0)) +
  scale_y_continuous("Stroke per 10^4 patients",
                     limits = c(0, 400),
                     expand = c(0, 0)) +
  ggtitle("Fear of Bleeding: 80/100\nGREEN = MEDICATE") +
  theme_bw() +
  theme(plot.title = element_text(hjust = 0.5))
print(p1)

4.6 Plot Fear Bleeding 80 on 0-100 Original Scale

p2 <- ggplot(tt,
             aes(x = pb,
                 y = ps,
                 fill = lab)) +
  geom_errorbar(aes(ymin = 10^4*(prob.stroke - 1.96*se.stroke),
                    ymax = 10^4*(prob.stroke + 1.96*se.stroke)),
                size = 0.5) +
  geom_errorbar(aes(xmin = 10^4*(prob.bleed - 1.96*se.bleed),
                    xmax = 10^4*(prob.bleed + 1.96*se.bleed)),
                size = 0.5) +
  geom_point(shape = 21,
             aes(size = sqrt(N))) +
  geom_ribbon(data = pi_20_80,
              aes(x = 2^l2pb,
                  ymin = 2^upr,
                  ymax = 400),
              fill = "green",
              alpha = 0.2,
              inherit.aes = FALSE) + 
  geom_line(data = pi_20_80,
            aes(x = 2^l2pb,
                y = 2^fit),
            linetype = "dashed",
            inherit.aes = FALSE) +
  scale_x_continuous("Bleeding per 10^4 patients",
                     limits = c(50, 400),
                     expand = c(0, 0)) +
  scale_y_continuous("Stroke per 10^4 patients",
                     limits = c(0, 400),
                     expand = c(0, 0)) +
  ggtitle("Fear of Bleeding: 80/100\nGREEN = MEDICATE") +
  theme_bw() +
  theme(plot.title = element_text(hjust = 0.5))
print(p2)

5 Method 2: Deming Regression

NOTE1: Deming regression requires standard errors for each point estimate. I do not have these estimates on the log scale. Hence, for the demonstration purposes, I’m showing regression models on the original scale.

NOTE2: how to do Deming regression prediction interval?

m1 <- lm(prob.stroke ~ prob.bleed,
         data = tt)
summary(m1)

Call:
lm(formula = prob.stroke ~ prob.bleed, data = tt)

Residuals:
       Min         1Q     Median         3Q        Max 
-0.0060634 -0.0024344 -0.0001482  0.0016504  0.0059096 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.002773   0.002342  -1.184 0.263725    
prob.bleed   0.688676   0.124877   5.515 0.000256 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.003809 on 10 degrees of freedom
Multiple R-squared:  0.7526,    Adjusted R-squared:  0.7278 
F-statistic: 30.41 on 1 and 10 DF,  p-value: 0.0002564
m2 <- deming(prob.stroke ~ prob.bleed,
             ystd = se.stroke,
             xstd = se.bleed,
             data = tt)
m2

Call:
deming(formula = prob.stroke ~ prob.bleed, data = tt, xstd = se.bleed,     ystd = se.stroke)

n= 12
                  Coef    se(coef)   lower 0.95   upper 0.95
Intercept -0.003480583 0.001783199 -0.006975588 1.442144e-05
Slope      0.726042116 0.218915292  0.296976028 1.155108e+00

   Scale= 2.014618 
p3 <- ggplot(tt,
             aes(x = prob.bleed,
                 y = prob.stroke,
                 fill = lab)) +
  geom_errorbar(aes(ymin = prob.stroke - 1.96*se.stroke,
                    ymax = prob.stroke + 1.96*se.stroke),
                size = 0.5) +
  geom_errorbar(aes(xmin = prob.bleed - 1.96*se.bleed,
                    xmax = prob.bleed + 1.96*se.bleed),
                size = 0.5) +
  geom_point(shape = 21,
             aes(size = sqrt(N))) +
  geom_abline(intercept = m1$coefficients[1],
              slope = m1$coefficients[2]) +
  geom_abline(intercept = m2$coefficients[1],
              slope = m2$coefficients[2],
              linetype = "dashed") +
  ggtitle("Least Squares (solid line) vs.\n Deming (dashed line) Regression") +
  theme_bw() +
  theme(plot.title = element_text(hjust = 0.5))
print(p3)

6 Session Information

sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] MethComp_1.30.0   deming_1.4        plotly_4.9.3      ggplot2_3.3.3    
[5] data.table_1.13.6

loaded via a namespace (and not attached):
 [1] pillar_1.4.7      compiler_4.0.3    tools_4.0.3       boot_1.3-25      
 [5] digest_0.6.27     evaluate_0.14     jsonlite_1.7.2    lifecycle_0.2.0  
 [9] tibble_3.0.5      gtable_0.3.0      nlme_3.1-151      viridisLite_0.3.0
[13] lattice_0.20-41   pkgconfig_2.0.3   rlang_0.4.10      cli_2.2.0        
[17] rstudioapi_0.13   crosstalk_1.1.1   yaml_2.2.1        xfun_0.20        
[21] coda_0.19-4       withr_2.4.0       dplyr_1.0.3       httr_1.4.2       
[25] knitr_1.30        generics_0.1.0    vctrs_0.3.6       htmlwidgets_1.5.3
[29] grid_4.0.3        tidyselect_1.1.0  glue_1.4.2        R6_2.5.0         
[33] fansi_0.4.2       rmarkdown_2.6     farver_2.0.3      purrr_0.3.4      
[37] tidyr_1.1.2       magrittr_2.0.1    scales_1.1.1      ellipsis_0.3.1   
[41] htmltools_0.5.0   assertthat_0.2.1  colorspace_2.0-0  labeling_0.4.2   
[45] lazyeval_0.2.2    munsell_0.5.0     crayon_1.3.4     
LS0tDQp0aXRsZTogIlN0cm9rZSB2cy4gQmxlZWRpbmcgaW4gTUlEQVMgUGF0aWVudHMgd2l0aGluIGEgWWVhciBvZiBJbml0aWFsIEFGIERpYWdub3NpcyINCm91dHB1dDoNCiAgaHRtbF9ub3RlYm9vazoNCiAgICBoaWdobGlnaHQ6IHRhbmdvDQogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMNCiAgICB0aGVtZTogdW5pdGVkDQogICAgdG9jOiBUUlVFDQogICAgdG9jX2Zsb2F0OiBUUlVFDQogICAgY29kZV9mb2xkaW5nOiBoaWRlDQotLS0NCg0KIyBTdHJva2UgdnMuIEJsZWVkaW5nIFNoaW55IEFwcCBEZXNpZ24gUHJvcG9zYWwNCiFbU3Ryb2tlIHZzLiBCbGVlZGluZyBTaGlueSBBcHAgRGVzaWduIFByb3Bvc2FsXShtZWRpYS9zdHJva2VfdnNfYmxlZWRfYXBwX2Rlc2lnbi5qcGcpIA0KIyBUbyBEbw0KMS4gV29yayB3aXRoIGEgc3RhdHMgc3R1ZGVudCAoWWF5aWUgRHVhbikgdG8gd3JpdGUgYSBTaGlueSBhcHAuICANCiAgDQoyLiBBZnRlciBtZWV0aW5nIHdpdGggSmF2aWVyICgwMS8yOC8yMDIxKTogIA0KICBhLiBJbnN0ZWFkIG9mIGxpbmVhciByZWdyZXNzaW9uLCB1c2UgW0RlbWluZyByZWdyZXNzaW9uXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9EZW1pbmdfcmVncmVzc2lvbikgIA0KICBiLiBQcm9iYWJpbGl0eSBvZiBiZWluZyBpbiB0aGUgZ3JlZW4gcmVnaW9uIGJhc2VkIG9uIHRoZSBjb25maWRlbmNlIGludGVydmFscyAodGFrZW4gY2FyZSBvZiBEZW1pbmcgcmVncmVzc2lvbj8pICANCiAgYy4gVGhpbmsgYWJvdXQ6IHJhdGlvIG9mIHZhcmlhbmNlIChyZWxhdGVkIHRvIERlbWluZyByZWdyZXNzaW9uKSAgDQogIA0KMy4gQWZ0ZXIgbWVldGluZyB3aXRoIENhbmRlICgwMS8yOC8yMDIxKTogIA0KICBhLiBBZGQgbW9yZSB2YXJpYWJsZXMgdG8gdGhlIG1vZGVsLiBJJ20gdGhpbmtpbmcgb2YgYWRkaW5nIENIQURTVkFTQyB2YXJpYWJsZXMgYW5kIGRpc3BsYXlpbmcgdGhlIHNjb3JlIGFkZGl0aW9uYWwgdG8gdGhlIHJlY29tbWVuZGF0aW9uLiBXZSBjYW4gYWxzbyBleHRlbmQgaXQgdG8gYSBtb3JlIGNvbXByZWhlbnNpdmUgbW9kZWwgYW5kIHVzZSB0aGUgYXBwIHRvIGFsc28gZGVtb25zdHJhdGUgaG93IHRoZSBNSURBUy1iYXNlZCByaXNrIGNhbGN1bGF0b3IgY29tcGFyZXMgdG8gQ0hBRFNWQVNDIHJpc2sgZXN0aW1hdGlvbi4gIA0KICAgIGIuIElGIHdlIGFyZSBnb2luZyB0byBhc3NpZ24gcHJvYmFiaWxpdGllcyBiYXNlZCBvbiBjb25maWRlbmNlIGludGVydmFscyAoc2VlICdiJyBhYm92ZSksIHdlIGNhbiBoYXZlIGFuIGludGVybmFsIHJ1bGUsIGUuZy4gaWYgcCA+IDAuNSB0aGVuIG1lZGljYXRlLCBpZiA8IHRoZW4gZG9uJ3QpLiBUaGlzIGRvZXMgbm90IG5lZWQgdG8gYmUgZGlzcGxheWVkIGluIHRoZSBhcHAgdG8gbWFrZSBpdCBsZXNzIGNvbmZ1c2luZyBmb3IgdGhlIHBhdGllbnRzLiAgDQogICAgDQojIFNldHVwDQpgYGB7ciBzZXR1cH0NCnJlcXVpcmUoZGF0YS50YWJsZSkNCnJlcXVpcmUoZ2dwbG90MikNCnJlcXVpcmUocGxvdGx5KQ0KcmVxdWlyZShkZW1pbmcpDQpyZXF1aXJlKE1ldGhDb21wKQ0KYGBgDQoNCioqQVRURU5USU9OISoqIEluIGdncGxvdCwgbXVzdCBleHBsaWNpdGx5IHNldCAnaW5oZXJpdC5hZXMgPSBGQUxTRScsIG90aGVyd2lzZSAnZ2VvbV9yaWJib24nIGV4cGVjdHMgdmFsdWVzIGZvciBhZXN0aGV0aWNzIGZyb20gcHJldmlvdXMgZ2VvbXMgKGkuZS4gdHQgdmFyaWFibGVzIHBzLCBwYiwgZXRjLikuIFNlZSBzb2x1dGlvbiBoZXJlOiAgDQooZ2VvbV9yaWJib24gZXhwZWN0cyBhbiB1bnVzZWQgeS1heGlzIHZhcmlhYmxlIHRvIGJlIHByZXNlbnQpW2h0dHBzOi8vZ2l0aHViLmNvbS90aWR5dmVyc2UvZ2dwbG90Mi9pc3N1ZXMvMzM2NF0NCg0KIyBMb2FkIGRhdGENCmBgYHtyIGRhdGF9DQpsb2FkKCJkYXRhL3R0LlJEYXRhIikNCmBgYA0KDQojIE1ldG9kIDE6IExpbmVhciByZWdyZXNzaW9uDQojIyBSdW4gJ2xtJw0KYGBge3IgbG19DQojIExvZzItdHJhbnNmb3JtZWQgdmFyaWFibGVzDQp0dFssIGMoImwycGIiLA0KICAgICAgICJsMnBzIikgOj0gbGlzdChsb2cyKHBiKSwgI3h4DQogICAgICAgICAgICAgICAgICAgICAgICBsb2cyKHBzKSldICN5eQ0KIyBSZWdyZXNzaW9uIG9uIGxvZzItdHJhbnNmb3JtZWQgdmFyaWFibGVzDQptMiA8LSBsbShsMnBzIH4gbDJwYiwNCiAgICAgICAgIGRhdGEgPSB0dCkNCnN1bW1hcnkobTIpDQoNCm5ld19sMnBiIDwtIHNlcShmcm9tID0gNS41LA0KICAgICAgICAgICAgICAgIHRvID0gOC43NSwNCiAgICAgICAgICAgICAgICBsZW5ndGgub3V0ID0gbnJvdyh0dCkpDQojIG5ld19sMnBiIDwtIHNlcShmcm9tID0gNSwgDQojICAgICAgICAgICAgICAgICB0byA9IDksDQojICAgICAgICAgICAgICAgICBsZW5ndGgub3V0ID0gbnJvdyh0dCkpDQpwaV8yMF84MCA8LSBkYXRhLnRhYmxlKHByZWRpY3QobTIsIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5ld2RhdGEgPSBsaXN0KGwycGIgPSBuZXdfbDJwYiksDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ZXJ2YWwgPSAicHJlZGljdGlvbiIsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGV2ZWwgPSAwLjYpKQ0KcGlfMjBfODAkbDJwYiA8LSBuZXdfbDJwYg0KaGVhZChwaV8yMF84MCkNCmBgYA0KDQojIyBQbG90IEZlYXIgQmxlZWRpbmcgMjAgb24gMC0xMDAgU2NhbGUgKExvZzIpDQpgYGB7ciBwbG90XzIwXzgwLGZpZy53aWR0aD03LGZpZy5oZWlnaHQ9Nn0NCnAxIDwtIGdncGxvdCh0dCwNCiAgICAgICAgICAgICBhZXMoeCA9IGwycGIsDQogICAgICAgICAgICAgICAgIHkgPSBsMnBzLA0KICAgICAgICAgICAgICAgICBmaWxsID0gbGFiKSkgKw0KICBnZW9tX2Vycm9yYmFyKGFlcyh5bWluID0gbG9nMigxMF40Kihwcm9iLnN0cm9rZSAtIDEuOTYqc2Uuc3Ryb2tlKSksDQogICAgICAgICAgICAgICAgICAgIHltYXggPSBsb2cyKDEwXjQqKHByb2Iuc3Ryb2tlICsgMS45NipzZS5zdHJva2UpKSksDQogICAgICAgICAgICAgICAgc2l6ZSA9IDAuNSkgKw0KICBnZW9tX2Vycm9yYmFyKGFlcyh4bWluID0gbG9nMigxMF40Kihwcm9iLmJsZWVkIC0gMS45NipzZS5ibGVlZCkpLA0KICAgICAgICAgICAgICAgICAgICB4bWF4ID0gbG9nMigxMF40Kihwcm9iLmJsZWVkICsgMS45NipzZS5ibGVlZCkpKSwNCiAgICAgICAgICAgICAgICBzaXplID0gMC41KSArDQogIGdlb21fcG9pbnQoc2hhcGUgPSAyMSwNCiAgICAgICAgICAgICBhZXMoc2l6ZSA9IHNxcnQoTikpKSArDQogIGdlb21fYWJsaW5lKHNsb3BlID0gbTIkY29lZmZpY2llbnRzWzJdLA0KICAgICAgICAgICAgICBpbnRlcmNlcHQgPSBtMiRjb2VmZmljaWVudHNbMV0sDQogICAgICAgICAgICAgIGxpbmV0eXBlID0gImRhc2hlZCIpICsNCiAgZ2VvbV9yaWJib24oZGF0YSA9IHBpXzIwXzgwLA0KICAgICAgICAgICAgICBhZXMoeCA9IGwycGIsDQogICAgICAgICAgICAgICAgICB5bWluID0gbHdyLA0KICAgICAgICAgICAgICAgICAgeW1heCA9IDkuMjUpLA0KICAgICAgICAgICAgICBmaWxsID0gImdyZWVuIiwNCiAgICAgICAgICAgICAgYWxwaGEgPSAwLjIsDQogICAgICAgICAgICAgIGluaGVyaXQuYWVzID0gRkFMU0UpICsgDQogIHNjYWxlX3hfY29udGludW91cygiTG9nMiBvZiBCbGVlZGluZyBwZXIgMTBeNCBwYXRpZW50cyIsDQogICAgICAgICAgICAgICAgICAgICAgIyBsaW1pdHMgPSBjKDUuNSwgOC41KSwNCiAgICAgICAgICAgICAgICAgICAgIGV4cGFuZCA9IGMoMCwgMCkpICsNCiAgc2NhbGVfeV9jb250aW51b3VzKCJMb2cyIG9mIFN0cm9rZSBwZXIgMTBeNCBwYXRpZW50cyIsDQogICAgICAgICAgICAgICAgICAgICAjIGxpbWl0cyA9IGMoMywgOSksDQogICAgICAgICAgICAgICAgICAgICBleHBhbmQgPSBjKDAsIDApKSArDQogIGdndGl0bGUoIkZlYXIgb2YgQmxlZWRpbmc6IDIwLzEwMFxuR1JFRU4gPSBNRURJQ0FURSIpICsNCiAgdGhlbWVfYncoKSArDQogIHRoZW1lKHBsb3QudGl0bGUgPSBlbGVtZW50X3RleHQoaGp1c3QgPSAwLjUpKQ0KZ2dwbG90bHkocDEpDQpgYGANCg0KIyMgUGxvdCBGZWFyIEJsZWVkaW5nIDgwIG9uIDAtMTAwIFNjYWxlIChMb2cyKQ0KYGBge3IgcGxvdF84MF8yMCxmaWcud2lkdGg9NyxmaWcuaGVpZ2h0PTZ9DQpwMiA8LSBnZ3Bsb3QodHQsDQogICAgICAgICAgICAgYWVzKHggPSBsMnBiLA0KICAgICAgICAgICAgICAgICB5ID0gbDJwcywNCiAgICAgICAgICAgICAgICAgZmlsbCA9IGxhYikpICsNCiAgZ2VvbV9lcnJvcmJhcihhZXMoeW1pbiA9IGxvZzIoMTBeNCoocHJvYi5zdHJva2UgLSAxLjk2KnNlLnN0cm9rZSkpLA0KICAgICAgICAgICAgICAgICAgICB5bWF4ID0gbG9nMigxMF40Kihwcm9iLnN0cm9rZSArIDEuOTYqc2Uuc3Ryb2tlKSkpLA0KICAgICAgICAgICAgICAgIHNpemUgPSAwLjUpICsNCiAgZ2VvbV9lcnJvcmJhcihhZXMoeG1pbiA9IGxvZzIoMTBeNCoocHJvYi5ibGVlZCAtIDEuOTYqc2UuYmxlZWQpKSwNCiAgICAgICAgICAgICAgICAgICAgeG1heCA9IGxvZzIoMTBeNCoocHJvYi5ibGVlZCArIDEuOTYqc2UuYmxlZWQpKSksDQogICAgICAgICAgICAgICAgc2l6ZSA9IDAuNSkgKw0KICBnZW9tX3BvaW50KHNoYXBlID0gMjEsDQogICAgICAgICAgICAgYWVzKHNpemUgPSBzcXJ0KE4pKSkgKw0KICBnZW9tX2FibGluZShzbG9wZSA9IG0yJGNvZWZmaWNpZW50c1syXSwNCiAgICAgICAgICAgICAgaW50ZXJjZXB0ID0gbTIkY29lZmZpY2llbnRzWzFdLA0KICAgICAgICAgICAgICBsaW5ldHlwZSA9ICJkYXNoZWQiKSArDQogIGdlb21fcmliYm9uKGRhdGEgPSBwaV8yMF84MCwNCiAgICAgICAgICAgICAgYWVzKHggPSBsMnBiLA0KICAgICAgICAgICAgICAgICAgeW1pbiA9IHVwciwNCiAgICAgICAgICAgICAgICAgIHltYXggPSA5LjI1KSwNCiAgICAgICAgICAgICAgZmlsbCA9ICJncmVlbiIsDQogICAgICAgICAgICAgIGFscGhhID0gMC4yLA0KICAgICAgICAgICAgICBpbmhlcml0LmFlcyA9IEZBTFNFKSArIA0KICBzY2FsZV94X2NvbnRpbnVvdXMoIkxvZzIgb2YgQmxlZWRpbmcgcGVyIDEwXjQgcGF0aWVudHMiLA0KICAgICAgICAgICAgICAgICAgICAgZXhwYW5kID0gYygwLCAwKSkgKw0KICBzY2FsZV95X2NvbnRpbnVvdXMoIkxvZzIgb2YgU3Ryb2tlIHBlciAxMF40IHBhdGllbnRzIiwNCiAgICAgICAgICAgICAgICAgICAgIGV4cGFuZCA9IGMoMCwgMCkpICsNCiAgZ2d0aXRsZSgiRmVhciBvZiBCbGVlZGluZzogODAvMTAwXG5HUkVFTiA9IE1FRElDQVRFIikgKw0KICB0aGVtZV9idygpICsNCiAgdGhlbWUocGxvdC50aXRsZSA9IGVsZW1lbnRfdGV4dChoanVzdCA9IDAuNSkpDQpnZ3Bsb3RseShwMikNCmBgYA0KDQojIyBSZWFkanVzdCBwcmVkaWN0aW9ucw0KYGBge3IgbG1fYW50aWxvZyxmaWcud2lkdGg9OCxmaWcuaGVpZ2h0PTZ9DQpuZXdfbDJwYiA8LSBzZXEoZnJvbSA9IDUsDQogICAgICAgICAgICAgICAgdG8gPSA5LA0KICAgICAgICAgICAgICAgIGxlbmd0aC5vdXQgPSAxMCpucm93KHR0KSkNCnBpXzIwXzgwIDwtIGRhdGEudGFibGUocHJlZGljdChtMiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbmV3ZGF0YSA9IGxpc3QobDJwYiA9IG5ld19sMnBiKSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnRlcnZhbCA9ICJwcmVkaWN0aW9uIiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsZXZlbCA9IDAuNikpDQpwaV8yMF84MCRsMnBiIDwtIG5ld19sMnBiDQpoZWFkKHBpXzIwXzgwKQ0KYGBgDQoNCiMjIFBsb3QgRmVhciBCbGVlZGluZyAyMCBvbiAwLTEwMCBPcmlnaW5hbCBTY2FsZQ0KYGBge3IgcGxvdF9hbnRpbG9nXzIwXzgwLGZpZy53aWR0aD03LGZpZy5oZWlnaHQ9Nn0NCnAxIDwtIGdncGxvdCh0dCwNCiAgICAgICAgICAgICBhZXMoeCA9IHBiLA0KICAgICAgICAgICAgICAgICB5ID0gcHMsDQogICAgICAgICAgICAgICAgIGZpbGwgPSBsYWIpKSArDQogIGdlb21fZXJyb3JiYXIoYWVzKHltaW4gPSAxMF40Kihwcm9iLnN0cm9rZSAtIDEuOTYqc2Uuc3Ryb2tlKSwNCiAgICAgICAgICAgICAgICAgICAgeW1heCA9IDEwXjQqKHByb2Iuc3Ryb2tlICsgMS45NipzZS5zdHJva2UpKSwNCiAgICAgICAgICAgICAgICBzaXplID0gMC41KSArDQogIGdlb21fZXJyb3JiYXIoYWVzKHhtaW4gPSAxMF40Kihwcm9iLmJsZWVkIC0gMS45NipzZS5ibGVlZCksDQogICAgICAgICAgICAgICAgICAgIHhtYXggPSAxMF40Kihwcm9iLmJsZWVkICsgMS45NipzZS5ibGVlZCkpLA0KICAgICAgICAgICAgICAgIHNpemUgPSAwLjUpICsNCiAgZ2VvbV9wb2ludChzaGFwZSA9IDIxLA0KICAgICAgICAgICAgIGFlcyhzaXplID0gc3FydChOKSkpICsNCiAgZ2VvbV9yaWJib24oZGF0YSA9IHBpXzIwXzgwLA0KICAgICAgICAgICAgICBhZXMoeCA9IDJebDJwYiwNCiAgICAgICAgICAgICAgICAgIHltaW4gPSAyXmx3ciwNCiAgICAgICAgICAgICAgICAgIHltYXggPSA0MDApLA0KICAgICAgICAgICAgICBmaWxsID0gImdyZWVuIiwNCiAgICAgICAgICAgICAgYWxwaGEgPSAwLjIsDQogICAgICAgICAgICAgIGluaGVyaXQuYWVzID0gRkFMU0UpICsgDQogIGdlb21fbGluZShkYXRhID0gcGlfMjBfODAsDQogICAgICAgICAgICBhZXMoeCA9IDJebDJwYiwNCiAgICAgICAgICAgICAgICB5ID0gMl5maXQpLA0KICAgICAgICAgICAgbGluZXR5cGUgPSAiZGFzaGVkIiwNCiAgICAgICAgICAgIGluaGVyaXQuYWVzID0gRkFMU0UpICsNCiAgc2NhbGVfeF9jb250aW51b3VzKCJCbGVlZGluZyBwZXIgMTBeNCBwYXRpZW50cyIsDQogICAgICAgICAgICAgICAgICAgICBsaW1pdHMgPSBjKDUwLCA0MDApLA0KICAgICAgICAgICAgICAgICAgICAgZXhwYW5kID0gYygwLCAwKSkgKw0KICBzY2FsZV95X2NvbnRpbnVvdXMoIlN0cm9rZSBwZXIgMTBeNCBwYXRpZW50cyIsDQogICAgICAgICAgICAgICAgICAgICBsaW1pdHMgPSBjKDAsIDQwMCksDQogICAgICAgICAgICAgICAgICAgICBleHBhbmQgPSBjKDAsIDApKSArDQogIGdndGl0bGUoIkZlYXIgb2YgQmxlZWRpbmc6IDgwLzEwMFxuR1JFRU4gPSBNRURJQ0FURSIpICsNCiAgdGhlbWVfYncoKSArDQogIHRoZW1lKHBsb3QudGl0bGUgPSBlbGVtZW50X3RleHQoaGp1c3QgPSAwLjUpKQ0KcHJpbnQocDEpDQpgYGANCg0KDQojIyBQbG90IEZlYXIgQmxlZWRpbmcgODAgb24gMC0xMDAgT3JpZ2luYWwgU2NhbGUNCmBgYHtyIHBsb3RfYW50aWxvZ184MF8yMCxmaWcud2lkdGg9NyxmaWcuaGVpZ2h0PTZ9DQpwMiA8LSBnZ3Bsb3QodHQsDQogICAgICAgICAgICAgYWVzKHggPSBwYiwNCiAgICAgICAgICAgICAgICAgeSA9IHBzLA0KICAgICAgICAgICAgICAgICBmaWxsID0gbGFiKSkgKw0KICBnZW9tX2Vycm9yYmFyKGFlcyh5bWluID0gMTBeNCoocHJvYi5zdHJva2UgLSAxLjk2KnNlLnN0cm9rZSksDQogICAgICAgICAgICAgICAgICAgIHltYXggPSAxMF40Kihwcm9iLnN0cm9rZSArIDEuOTYqc2Uuc3Ryb2tlKSksDQogICAgICAgICAgICAgICAgc2l6ZSA9IDAuNSkgKw0KICBnZW9tX2Vycm9yYmFyKGFlcyh4bWluID0gMTBeNCoocHJvYi5ibGVlZCAtIDEuOTYqc2UuYmxlZWQpLA0KICAgICAgICAgICAgICAgICAgICB4bWF4ID0gMTBeNCoocHJvYi5ibGVlZCArIDEuOTYqc2UuYmxlZWQpKSwNCiAgICAgICAgICAgICAgICBzaXplID0gMC41KSArDQogIGdlb21fcG9pbnQoc2hhcGUgPSAyMSwNCiAgICAgICAgICAgICBhZXMoc2l6ZSA9IHNxcnQoTikpKSArDQogIGdlb21fcmliYm9uKGRhdGEgPSBwaV8yMF84MCwNCiAgICAgICAgICAgICAgYWVzKHggPSAyXmwycGIsDQogICAgICAgICAgICAgICAgICB5bWluID0gMl51cHIsDQogICAgICAgICAgICAgICAgICB5bWF4ID0gNDAwKSwNCiAgICAgICAgICAgICAgZmlsbCA9ICJncmVlbiIsDQogICAgICAgICAgICAgIGFscGhhID0gMC4yLA0KICAgICAgICAgICAgICBpbmhlcml0LmFlcyA9IEZBTFNFKSArIA0KICBnZW9tX2xpbmUoZGF0YSA9IHBpXzIwXzgwLA0KICAgICAgICAgICAgYWVzKHggPSAyXmwycGIsDQogICAgICAgICAgICAgICAgeSA9IDJeZml0KSwNCiAgICAgICAgICAgIGxpbmV0eXBlID0gImRhc2hlZCIsDQogICAgICAgICAgICBpbmhlcml0LmFlcyA9IEZBTFNFKSArDQogIHNjYWxlX3hfY29udGludW91cygiQmxlZWRpbmcgcGVyIDEwXjQgcGF0aWVudHMiLA0KICAgICAgICAgICAgICAgICAgICAgbGltaXRzID0gYyg1MCwgNDAwKSwNCiAgICAgICAgICAgICAgICAgICAgIGV4cGFuZCA9IGMoMCwgMCkpICsNCiAgc2NhbGVfeV9jb250aW51b3VzKCJTdHJva2UgcGVyIDEwXjQgcGF0aWVudHMiLA0KICAgICAgICAgICAgICAgICAgICAgbGltaXRzID0gYygwLCA0MDApLA0KICAgICAgICAgICAgICAgICAgICAgZXhwYW5kID0gYygwLCAwKSkgKw0KICBnZ3RpdGxlKCJGZWFyIG9mIEJsZWVkaW5nOiA4MC8xMDBcbkdSRUVOID0gTUVESUNBVEUiKSArDQogIHRoZW1lX2J3KCkgKw0KICB0aGVtZShwbG90LnRpdGxlID0gZWxlbWVudF90ZXh0KGhqdXN0ID0gMC41KSkNCnByaW50KHAyKQ0KYGBgDQoNCiMgTWV0aG9kIDI6IERlbWluZyBSZWdyZXNzaW9uDQoqTk9URTEqOiBEZW1pbmcgcmVncmVzc2lvbiByZXF1aXJlcyBzdGFuZGFyZCBlcnJvcnMgZm9yIGVhY2ggcG9pbnQgZXN0aW1hdGUuIEkgZG8gbm90IGhhdmUgdGhlc2UgZXN0aW1hdGVzIG9uIHRoZSBsb2cgc2NhbGUuIEhlbmNlLCBmb3IgdGhlIGRlbW9uc3RyYXRpb24gcHVycG9zZXMsIEknbSBzaG93aW5nIHJlZ3Jlc3Npb24gbW9kZWxzIG9uIHRoZSBvcmlnaW5hbCBzY2FsZS4gIA0KICANCipOT1RFMio6IGhvdyB0byBkbyBEZW1pbmcgcmVncmVzc2lvbiBwcmVkaWN0aW9uIGludGVydmFsPyAgDQogIA0KYGBge3IgZGVtaW5nLGZpZy53aWR0aD03LGZpZy5oZWlnaHQ9Nn0NCm0xIDwtIGxtKHByb2Iuc3Ryb2tlIH4gcHJvYi5ibGVlZCwNCiAgICAgICAgIGRhdGEgPSB0dCkNCnN1bW1hcnkobTEpDQoNCm0yIDwtIGRlbWluZyhwcm9iLnN0cm9rZSB+IHByb2IuYmxlZWQsDQogICAgICAgICAgICAgeXN0ZCA9IHNlLnN0cm9rZSwNCiAgICAgICAgICAgICB4c3RkID0gc2UuYmxlZWQsDQogICAgICAgICAgICAgZGF0YSA9IHR0KQ0KbTINCg0KcDMgPC0gZ2dwbG90KHR0LA0KICAgICAgICAgICAgIGFlcyh4ID0gcHJvYi5ibGVlZCwNCiAgICAgICAgICAgICAgICAgeSA9IHByb2Iuc3Ryb2tlLA0KICAgICAgICAgICAgICAgICBmaWxsID0gbGFiKSkgKw0KICBnZW9tX2Vycm9yYmFyKGFlcyh5bWluID0gcHJvYi5zdHJva2UgLSAxLjk2KnNlLnN0cm9rZSwNCiAgICAgICAgICAgICAgICAgICAgeW1heCA9IHByb2Iuc3Ryb2tlICsgMS45NipzZS5zdHJva2UpLA0KICAgICAgICAgICAgICAgIHNpemUgPSAwLjUpICsNCiAgZ2VvbV9lcnJvcmJhcihhZXMoeG1pbiA9IHByb2IuYmxlZWQgLSAxLjk2KnNlLmJsZWVkLA0KICAgICAgICAgICAgICAgICAgICB4bWF4ID0gcHJvYi5ibGVlZCArIDEuOTYqc2UuYmxlZWQpLA0KICAgICAgICAgICAgICAgIHNpemUgPSAwLjUpICsNCiAgZ2VvbV9wb2ludChzaGFwZSA9IDIxLA0KICAgICAgICAgICAgIGFlcyhzaXplID0gc3FydChOKSkpICsNCiAgZ2VvbV9hYmxpbmUoaW50ZXJjZXB0ID0gbTEkY29lZmZpY2llbnRzWzFdLA0KICAgICAgICAgICAgICBzbG9wZSA9IG0xJGNvZWZmaWNpZW50c1syXSkgKw0KICBnZW9tX2FibGluZShpbnRlcmNlcHQgPSBtMiRjb2VmZmljaWVudHNbMV0sDQogICAgICAgICAgICAgIHNsb3BlID0gbTIkY29lZmZpY2llbnRzWzJdLA0KICAgICAgICAgICAgICBsaW5ldHlwZSA9ICJkYXNoZWQiKSArDQogIGdndGl0bGUoIkxlYXN0IFNxdWFyZXMgKHNvbGlkIGxpbmUpIHZzLlxuIERlbWluZyAoZGFzaGVkIGxpbmUpIFJlZ3Jlc3Npb24iKSArDQogIHRoZW1lX2J3KCkgKw0KICB0aGVtZShwbG90LnRpdGxlID0gZWxlbWVudF90ZXh0KGhqdXN0ID0gMC41KSkNCnByaW50KHAzKQ0KYGBgDQoNCiMgU2Vzc2lvbiBJbmZvcm1hdGlvbg0KYGBge3Igc2Vzc2lvbn0NCnNlc3Npb25JbmZvKCkNCmBgYA0K